Normalized entropy: A comparison with traditional techniques in variable selection

A variable selection procedure in regression analysis using a normalized entropy measure was firstly proposed 1996, by Amos Golan, George Judge and Douglas Miller, the book Maximum Entropy Econometrics – Robust Estimation with Limited Data. To best of authors’ knowledge, idea has not been explored literature since then, despite many noteworthy advantages that have pointed out Golan coauthors, such as: it is simple to perform, even for large number variables (useful some big data problems); allows use non-sample information (easily incorporated optimization structure); can be implemented ill-posed models (frequently observed real-world problems). Following recent work illustrates how represent promising approach identify pure noise models, this paper revises entropy, proposes improvements, its performance when compared well-known traditional techniques problems.